CxS Data Quality Calculus : Assessing Data Quality in Heterogeneous Databases

Organizations in various research, military, commercial, and governmental institutions have realized the importance of systems integration. As information systems are integrated and information highways established, quality of data flowing through these highways to data consumers becomes increasingly critical. A key, and largely unexplored, challenge lies in how to evaluate data quality. This paper attempts to ascertain, and represent, knowledge that allows assessing data quality with respect to a particular quality dimension. In addition, algorithms for computing such quality dimension values are investigated. In this research, for any quality dimension, its value is defined as a net effect of quality parameters that affect it. This research investigates the notion of a data quality calculus to combine effects of affecting quality parameters to a whole effect. The primary constructs of the data quality calculus consist of quality join and qualityjoin expression relations. The data quality calculus provides a mechanism via which data consumers can explicitly specify, according to their needs, relationships between quality parameters. In addition, a methodology for computing the quality join based on relationships between quality parameters is an integral part of the calculus. The explicit representation of affecting quality parameters and their relationships to each other greatly simplifies the algorithmic component to provide a data consumer with data quality information which is specifically tuned to the customer's needs. *Research conducted herein has been supported in part by the National Heart, Lung, and Blood Institute under the grant number R01 HL33041, in part by the National Institutes of Health under the grant number R01 LM04493 from the National Library of Medicine, and in part by the International Financial Services Research Center at MIT.